Techniques for Generating a Representative Image and Radiographic Interpretation Information for a Case

ABSTRACT

A technique for generating a representative image representing a case and radiographic interpretation information for each case includes calculating wavelet features of a plurality of images that have been taken and stored in the past. The calculated wavelet features and extracted keywords are stored in association with the stored images. The stored images are classified on the basis of the extracted keywords to generate a plurality of groups. For each of the generated groups, a centroid vector of wavelet feature-based feature vectors of respective images corresponding to the keywords is calculated and a spatial distance between the calculated centroid vector and each of the wavelet feature-based feature vectors is calculated. For each of the groups, the image for which the calculated spatial distance is the shortest and the radiographic interpretation information associated with the image is stored as a representative image of that group.

This application is a national stage of International Application No.PCT/JP2012/074077, entitled “METHOD, APPARATUS, AND COMPUTER PROGRAM FORGENERATING REPRESENTATIVE IMAGE AND RADIOGRAPHIC INTERPRETATIONINFORMATION FOR EACH CASE,” filed Sep. 20, 2012, which claims priorityto Japanese Patent Application No. 2011-272175, filed Dec. 13, 2011. Thedisclosure of International Application No. PCT/JP2012/074077 is herebyincorporated herein by reference in its entirety for all purposes.

BACKGROUND

The disclosure relates to image generation and, more specifically, togenerating representative image and radiographic interpretationinformation on the basis of a plurality of stored images for a case.

In medical practice, it is important to recognize the internal states ofa patient on the basis of images acquired by radiography. An X-ray imageof a patient can be compared with X-ray images taken in the past toidentify the causes of symptoms of the patient. This enables selectionof appropriate medical treatment, leading to early improvement of thesymptoms of the patient.

In the case of identifying a patient's symptoms from an X-ray image ofthe patient, images taken and stored in the past may be searched for asimilar image, so as to confirm which case looks similar to the symptomsof the patient. For example, Japanese Unexamined Patent Publication No.2007-279942 discloses a similar case search apparatus which searches fora similar case image and similar case data on the basis of featurevalues obtained from a diagnosis object image.

BRIEF SUMMARY

A technique for generating a representative image representing a caseand radiographic interpretation information for each case from medicalimages based on past cases includes calculating wavelet features of aplurality of images that have been taken and stored in the past. Akeyword included in radiographic interpretation information is extractedfor each of the stored images. The calculated wavelet features and theextracted keywords are stored in association with the respective storedimages. The stored images are classified on the basis of the extractedkeywords to generate a plurality of groups. For each of the generatedgroups, a centroid vector of wavelet feature-based feature vectors ofrespective images corresponding to the keywords included in that groupis calculated. For each of the groups, a spatial distance between thecalculated centroid vector and each of the wavelet feature-based featurevectors of the respective images corresponding to the keywords includedin that group is calculated. For each of the groups, the image for whichthe calculated spatial distance is the shortest and the radiographicinterpretation information associated with the image is stored as arepresentative image of that group.

The above summary contains simplifications, generalizations andomissions of detail and is not intended as a comprehensive descriptionof the claimed subject matter but, rather, is intended to provide abrief overview of some of the functionality associated therewith. Othersystems, methods, functionality, features and advantages of the claimedsubject matter will be or will become apparent to one with skill in theart upon examination of the following figures and detailed writtendescription.

The above as well as additional objectives, features, and advantages ofthe present invention will become apparent in the following detailedwritten description.

BRIEF DESCRIPTION OF THE DRAWINGS

The description of the illustrative embodiments is to be read inconjunction with the accompanying drawings, wherein:

FIG. 1 is a block diagram schematically showing the configuration of acase image generating apparatus according to an embodiment of thepresent disclosure;

FIG. 2 is a functional block diagram of a case image generatingapparatus according an embodiment of the present disclosure;

FIG. 3 illustrates coordinate setting within an image used in a caseimage generating apparatus according to an embodiment of the presentdisclosure;

FIGS. 4A and 4B show, by way of example, a two-dimensional Gabor waveletfunction;

FIG. 5 is a schematic diagram showing the directions of thetwo-dimensional Gabor wavelet function used in a case image generatingapparatus according to an embodiment of the present disclosure;

FIG. 6 shows, by way of example, the data structure of visual wordsstored in a visual word storage unit in a case image generatingapparatus according to an embodiment of the present disclosure;

FIG. 7 shows, by way of example, keyword extraction by a case imagegenerating apparatus according to an embodiment of the presentdisclosure;

FIG. 8 shows, by way of example, a histogram according to an embodimentof the present disclosure;

FIG. 9 shows an example of a representative image display screen used ina case image generating apparatus according to an embodiment of thepresent disclosure; and

FIG. 10 is a flowchart illustrating a processing procedure of a CPU in acase image generating apparatus according to an embodiment of thepresent disclosure.

DETAILED DESCRIPTION

The illustrative embodiments provide a method, an apparatus, and acomputer program product configured to generate a representative imageand radiographic interpretation information for each case on the basisof stored past images.

In the following detailed description of exemplary embodiments of theinvention, specific exemplary embodiments in which the invention may bepracticed are described in sufficient detail to enable those skilled inthe art to practice the invention, and it is to be understood that otherembodiments may be utilized and that logical, architectural,programmatic, mechanical, electrical and other changes may be madewithout departing from the spirit or scope of the present invention. Thefollowing detailed description is, therefore, not to be taken in alimiting sense, and the scope of the present invention is defined by theappended claims and equivalents thereof.

It should be understood that the use of specific component, device,and/or parameter names are for example only and not meant to imply anylimitations on the invention. The invention may thus be implemented withdifferent nomenclature/terminology utilized to describe thecomponents/devices/parameters herein, without limitation. Each termutilized herein is to be given its broadest interpretation given thecontext in which that term is utilized. As may be used herein, the term‘coupled’ may encompass a direct connection between components orelements or an indirect connection between components or elementsutilizing one or more intervening components or elements.

While the similar case search apparatus disclosed in Japanese UnexaminedPatent Publication No. 2007-279942 is capable of searching for an imagesimilar to the image of the patient, usually a plurality of similarimages are found. While the similar case search apparatus is able todisplay the images in descending order of similarity, a large number ofimages different in symptoms, though similar in image feature, arefound, and therefore the similar case search apparatus has a problemthat it might be difficult to identify the symptoms in some cases.

In addition, although the apparatus is able to search stored images thathave been classified according to cases, it is not possible to determinewhich one is a representative image, and therefore none of the valuablepast images can be used as a guideline image for a case.

The disclosed embodiments have been provided in view of the abovebackground and are configured to generate a representative image andradiographic interpretation information for each case on the basis ofstored past images.

A method according to the present disclosure is capable of beingexecuted in an apparatus to generate a representative image representinga case and radiographic interpretation information for each case frommedical images based on past cases. In at least one embodiment, themethod includes: calculating wavelet features of a plurality of imagesthat have been taken and stored in the past; extracting a keywordincluded in radiographic interpretation information for each storedimage; storing the calculated wavelet features and the extractedkeywords in association with the respective stored images; classifyingthe stored images on the basis of the extracted keywords to generate aplurality of groups; calculating, for each of the generated groups, acentroid vector of wavelet feature-based feature vectors of respectiveimages corresponding to the keywords included in that group;calculating, for each of the groups, a spatial distance between thecalculated centroid vector and each of the wavelet feature-based featurevectors of the respective images corresponding to the keywords includedin that group; and storing, for each of the groups, the image thecalculated spatial distance of which is the shortest and theradiographic interpretation information associated with the image, as arepresentative image of that group.

The method may calculate a two-dimensional Gabor wavelet feature as thewavelet feature.

The method may also include calculating frequency distribution vectorsfor all images, by calculating M said wavelet features (M is a naturalnumber of 2 or greater) for each image and binarizing the respectivewavelet features for conversion into an M-dimensional bit string. Thespatial distance may then be calculated as an angle between the centroidvector and each of the calculated frequency distribution vectors.

An apparatus configured according to the present disclosure may beconfigured to generate a representative image representing a case andradiographic interpretation information for each case from medicalimages based on past cases. The apparatus may include: a wavelet featurecalculation unit for calculating wavelet features of a plurality ofimages that have been taken and stored in the past; a keyword extractionunit for extracting a keyword included in radiographic interpretationinformation for each stored image; an information storage unit forstoring the calculated wavelet features and the extracted keywords inassociation with the respective stored images; a group generation unitfor classifying the stored images on the basis of the extracted keywordsto generate a plurality of groups; a centroid vector calculation unitfor calculating, for each of the generated groups, a centroid vector ofwavelet feature-based feature vectors of respective images correspondingto the keywords included in that group; a spatial distance calculationunit for calculating, for each of the groups, a spatial distance betweenthe calculated centroid vector and each of the wavelet feature-basedfeature vectors of the respective images corresponding to the keywordsincluded in that group; and a representative image storage unit forstoring, for each of the groups, the image the calculated spatialdistance of which is the shortest and the radiographic interpretationinformation associated with the image, as a representative image of thatgroup.

The wavelet feature calculation unit may calculate a two-dimensionalGabor wavelet feature as the wavelet feature.

The apparatus may further include a frequency distribution vectorcalculation unit for calculating frequency distribution vectors for allimages, by calculating M said wavelet features (M is a natural number of2 or greater) for each image and binarizing the respective waveletfeatures for conversion into an M-dimensional bit string. The spatialdistance calculation unit may then calculate the spatial distance as anangle between the centroid vector and each of the calculated frequencydistribution vectors.

A computer program executable by an apparatus may be configured togenerate a representative image representing a case and radiographicinterpretation information for each case, from medical images based onpast cases. The program may cause the apparatus to function as: awavelet feature calculation unit for calculating wavelet features of aplurality of images that have been taken and stored in the past; akeyword extraction unit for extracting a keyword included inradiographic interpretation information for each stored image; aninformation storage unit for storing the calculated wavelet features andthe extracted keywords in association with the respective stored images;a group generation unit for classifying the stored images on the basisof the extracted keywords to generate a plurality of groups; a centroidvector calculation unit for calculating, for each of the generatedgroups, a centroid vector of wavelet feature-based feature vectors ofrespective images corresponding to the keywords included in that group;a spatial distance calculation unit for calculating, for each of thegroups, a spatial distance between the calculated centroid vector andeach of the wavelet feature-based feature vectors of the respectiveimages corresponding to the keywords included in that group; and arepresentative image storage unit for storing, for each of the groups,the image the calculated spatial distance of which is the shortest andthe radiographic interpretation information associated with the image,as a representative image of that group.

The computer program may be configured to cause the wavelet featurecalculation unit to function as unit for calculating a two-dimensionalGabor wavelet feature as the wavelet feature.

The program may be further configured to cause the apparatus to functionas frequency distribution vector calculation unit for calculatingfrequency distribution vectors for all images, by calculating M saidwavelet features (M is a natural number of 2 or greater) for each imageand binarizing the respective wavelet features for conversion into anM-dimensional bit string. The program may cause the spatial distancecalculation unit to function as unit for calculating the spatialdistance as an angle between the centroid vector and each of thecalculated frequency distribution vectors.

According to the present disclosure, wavelet features indicating thefeatures of stored medical images are used to calculate feature vectorsfor the respective images, and from these feature vectors, a centroidvector is calculated as a feature vector for each case. Among the storedmedical images, an image having the feature vector with the shortestspatial distance from the centroid vector is stored as a representativeimage. Therefore, the image indicating a typical case can be used as aguideline of the case, allowing a diagnosis to be made for a patient ata certain level of quality, without being affected by experience orexpertise of each doctor.

A case image generating apparatus for generating a representative imageand radiographic interpretation information for each case out of medicalimages based on past cases according to an embodiment is specificallydescribed below with reference to the drawings. The followingembodiments do not restrict the claimed invention, and all thecombinations of the features described in the embodiment are notnecessarily indispensable.

Further, the present invention can be carried out in many differentmodes, and should not be understood only from the description given.Through the whole description of the embodiment, the same elements aredenoted by the same reference numerals.

While an apparatus comprising a computer system having a computerprogram introduced therein is described, it should be apparent to thoseskilled in the art that part of the present invention may be implementedas a computer-executable computer program. Therefore, the presentinvention can take the form of an embodiment as hardware which is a caseimage generating apparatus that generates a representative image andradiographic interpretation information for each case out of medicalimages based on past images or an embodiment as a combination ofsoftware and hardware. The computer program may be recorded on anarbitrary computer-readable recording medium, such as a hard disk, aDVD, a CD, an optical storage device, or a magnetic storage device.

According to one embodiment, wavelet features indicating the features ofstored medical images are used to calculate feature vectors for therespective images, and from these feature vectors, a centroid vector iscalculated as a feature vector for each case. Among the stored medicalimages, an image having the feature vector with the shortest spatialdistance from the centroid vector is stored as a representative image.Therefore, the image indicating a typical case can be used as aguideline of the case, allowing a diagnosis to be made for a patient ata certain level of quality, without being affected by experience orexpertise of each doctor.

FIG. 1 is a block diagram schematically showing the configuration of acase image generating apparatus (e.g., computer system) according to anembodiment of the present disclosure. The case image generatingapparatus 1 according to the embodiment at least includes: a centralprocessing unit (CPU) or processor 11, a memory 12, a storage device 13,an I/O interface 14, a video interface 15, a portable disk drive 16, acommunication interface 17, and an internal bus 18 for connecting theabove-described hardware components.

The CPU 11 is connected via the internal bus 18 to the hardwarecomponents of the case image generating apparatus 1 as described above.The CPU 11 controls the operations of those hardware components, andalso executes various software functions in accordance with a computerprogram 100 stored in the storage device 13. The memory 12 is made up ofa volatile memory such as an SRAM or an SDRAM, in which a load module isdeployed at the time of execution of the computer program 100. Temporarydata generated during the execution of the computer program 100 is alsostored in the memory 12.

The storage device 13 includes a built-in fixed storage (hard disk), aROM, and others. The computer program 100 stored in the storage device13 is one that has been downloaded by the portable disk drive 16 from aportable recording medium 90 such as a DVD or a CD-ROM that recordsinformation such as data and programs. At run-time, the computer program100 is deployed from the storage device 13 to the memory 12 forexecution. The computer program 100 may of course be downloaded from anexternal computer connected via the communication interface 17.

The storage device 13 includes a medical image storage unit 131, aradiographic interpretation information storage unit 132, a visual wordstorage unit 133, a frequency distribution information storage unit 134,and a case image database 135. The medical image storage unit 131 storesimage data of X-ray images taken in the past. The unit 131 stores theimage data in association with identification information foridentifying radiographic interpretation information.

The radiographic interpretation information storage unit 132 storesresults of diagnoses that doctors have made by interpreting medicalimages taken in the past. For example, a doctor's diagnosis such as“nodular shadow found in left lung field, upper lobe; squamous cellcarcinoma suspected; workup by HR-CT instructed” is stored in the formof text data in association with identification information.

The visual word storage unit 133 stores, as visual words, Gabor waveletfeatures which will be described later. The frequency distributioninformation storage unit 134 stores, as feature vectors, frequencydistribution vectors of values obtained by binarizing calculated waveletfeatures and converting them into M-dimensional bit strings.

The case image database 135 stores, for each case, a representativeimage as the most typical image for that case and radiographicinterpretation information corresponding to the representative image, inthe form of database. The case image database 135 functions asguidelines for cases, from which a typical image for each case can beextracted. This allows a diagnosis to be made for a patient at a certainlevel of quality, without being affected by experience or expertise ofeach doctor.

The communication interface 17 is connected to the internal bus 18, andto an external network such as the Internet, a LAN, or a WAN, so that itis able to transmit data to and receive data from an external computer.

The I/O interface 14 is connected to input devices such as a keyboard 21and a mouse 22, and accepts input of data. The video interface 15 isconnected to a display device 23 such as a CRT display or a liquidcrystal display, and displays a representative image and radiographicinterpretation information corresponding to the representative image onthe display device 23.

FIG. 2 is a functional block diagram of the case image generatingapparatus 1 according to one embodiment. Referring to FIG. 2, a waveletfeature calculation unit 201 in the case image generating apparatus 1calculates wavelet features of a plurality of images taken and stored inthe past. In one embodiment, Gabor wavelet features are calculated asthe wavelet features.

FIG. 3 illustrates coordinate setting within an image used in the caseimage generating apparatus 1 according to one embodiment. As shown inFIG. 3, an image with an origin at an upper left corner thereof andhaving m pixels in an x direction and n pixels in a y direction isdefined as s(x, y). The coordinates of an i-th pixel P, (i is a naturalnumber) are represented as P_(i)(x_(i), y_(i)).

First, the coordinates P_(i)(x_(i), y_(i)) are affine-transformed tocoordinates (X_(i), Y_(i)) in accordance with the following expression(1).

[X_(i),Y_(i),1]=[x_(i),y_(i),1]A  (1)

In the above expression (1), the matrix A is a 3×3 affine transformationmatrix. The affine transformation to shift the entire image by tx in thex direction and by ty in the y direction can be expressed by thefollowing expression (2), and the affine transformation to rotate theentire image by an angle θ can be expressed by the following expression(3).

$\begin{matrix}{\left\lbrack {X_{i},Y_{i},1} \right\rbrack = {\left\lbrack {x_{i},y_{i},1} \right\rbrack \begin{bmatrix}1 & 0 & 0 \\0 & 1 & 0 \\{tx} & {ty} & 1\end{bmatrix}}} & (2) \\{\left\lbrack {X_{i},Y_{i},1} \right\rbrack = {\left\lbrack {x_{i},y_{i},1} \right\rbrack \begin{bmatrix}{\cos \; \theta} & {\sin \; \theta} & 0 \\{{- \sin}\; \theta} & {\cos \; \theta} & 0 \\0 & 0 & 1\end{bmatrix}}} & (3)\end{matrix}$

A two-dimensional Gabor wavelet function is defined, with respect to thecoordinate values (x with dot, y with dot) after the affinetransformation for rotation, as in the following expression (4).

$\begin{matrix}{{\psi_{r}\left( {x,y} \right)} = {{{{g_{\sigma}\left( {\overset{.}{x},\overset{.}{y}} \right)}\left\lbrack {^{\; u_{0}\overset{.}{x}} - ^{- {({u_{0}\sigma})}^{2}}} \right\rbrack}\begin{bmatrix}\overset{.}{x} \\\overset{.}{y}\end{bmatrix}} = {\begin{bmatrix}{\cos \; \theta_{r}} & {\sin \; \theta_{r}} \\{{- \sin}\; \theta_{r}} & {\cos \; \theta_{r}}\end{bmatrix}\begin{bmatrix}x \\y\end{bmatrix}}}} & (4)\end{matrix}$

The two-dimensional Gabor wavelet function is composed of a real partand an imaginary part. FIGS. 4A and 4B show an example of atwo-dimensional Gabor wavelet function. Specifically, FIGS. 4A and 4Bshow examples of the real part and imaginary part, respectively, of thetwo-dimensional Gabor wavelet function. As seen from FIGS. 4A and 4B,the real part of the two-dimensional Gabor wavelet function has ahat-like wavy form with its maximum value located near (x, y)=(0, 0). Inthe above expression (4), u₀ represents the frequency of that waveform,and σ represents the width of that hat shape. Further, r represents thedirection, which will be described later.

The window function g_(σ) in the above expression (4) is atwo-dimensional Gaussian function, which can be expressed by thefollowing expression (5).

$\begin{matrix}{{g_{\sigma}\left( {\overset{.}{x},\overset{.}{y}} \right)} = {\frac{1}{4\pi \; \sigma}^{\frac{- 1}{4\sigma^{2}}{({{\overset{.}{x}}^{2} + {\overset{.}{y}}^{2}})}}}} & (5)\end{matrix}$

Using the two-dimensional Gabor wavelet function, the Gabor waveletfeatures for an acquired image s(x, y) can be calculated by thefollowing expression (6). The lattice point at which the absolute valueof the Gabor wavelet feature has a maximum value and the Gabor waveletfeatures in the vicinity of that lattice point are invariant even whenthe image is subjected to affine transformation such as scaling,rotation, etc., so that they are suitably used as the feature values ofan image.

$\begin{matrix}{{G_{j,r}\left( {x_{0},y_{0}} \right)} = {a^{- j}{\int{\int{{s\left( {x,y} \right)}{\psi_{r}\left( {\frac{x - x_{0}}{a^{j}},\frac{y - y_{0}}{a^{j}}} \right)}{x}{y}}}}}} & (6)\end{matrix}$

In the above expression (6), a^(j) and a^(−j) are parameters indicatingthe degrees of dilation (scaling), and x₀ and y₀ represent shift.Further, r represents the direction. In the present embodiment, theGabor wavelet features in eight directions are calculated.

FIG. 5 is a schematic diagram showing the directions of thetwo-dimensional Gabor wavelet function used in the case image generatingapparatus 1 according to one embodiment. As shown in FIG. 5, in thepresent embodiment, the Gabor wavelet features are calculated indirections (1) to (8), i.e., in eight directions spaced every 22.5degrees from a prescribed direction.

The calculation of the Gabor wavelet features makes it possible tocalculate the wavelet feature values that accommodate or absorbvariations in shape of the human organs, for example, so that a moreappropriate representative image can be selected from among the imagesrelated to the same case.

For example, in the case where the above expression (6) is used tocalculate the Gabor wavelet features for each coordinate point (x, y)(lattice point within an image), eight directions (r=1 to 8) and fivescales (j=1 to 5) are selected to calculate 40 Gabor wavelet featuresfor one coordinate point. Here, the scales 1 to 5 indicate the levels ofenlargement/reduction. For example, a greater value indicates a greaterdegree of enlargement. From the Gabor wavelet features calculated, thosehaving the absolute values of not less than a predetermined thresholdvalue are extracted, and the Gabor wavelet feature having a maximumvalue among them is selected.

The fact that the absolute value of the Gabor wavelet feature takes amaximum value unit that the absolute value of the integral in the aboveexpression (6) is maximum. The feature value remains unchanged even whenthe average brightness of the image is changed, the scale of the imageis changed, or the image is rotated.

In the present embodiment, the Gabor wavelet features in eightdirections in the scale where a maximum value is obtained, as well asthe Gabor wavelet features in the eight directions in each of theneighboring scales, namely 24 (3 scales×8 directions) Gabor waveletfeatures in total, are stored as a set of visual words in the visualword storage unit 133.

FIG. 6 shows, by way of example, the data structure of visual wordsstored in the visual word storage unit 133 in the storage device 13 inthe case image generating apparatus 1 according to one embodiment. Asshown in FIG. 6, 24 Gabor wavelet features which have been calculatedare listed and stored corresponding to each identification number 1, 2,3, . . . . More specifically, “1” at the beginning is the identificationnumber, which is followed by a blank space, and the numerical valuesfollowing “1:” to “24:” are the 24 Gabor wavelet features calculated.FIG. 6 shows the visual words in the case where there are three maximumvalues within one image. Thus, in FIG. 6, the visual words are storedcorresponding to three identification numbers “1”, “2”, and “3”. Whenthere is one maximum value, there is naturally only one identificationnumber “1”.

Returning to FIG. 2, a keyword extraction unit 202 extracts keywordsthat are included in the radiographic interpretation information storedin the radiographic interpretation information storage unit 132 in thestorage device 13 corresponding to the past images stored in the medicalimage storage unit 131 in the storage device 13. For example, in thecase where radiographic interpretation information reading: “nodularshadow found in left lung field, upper lobe; squamous cell carcinomasuspected; workup by HR-CT instructed” is stored in the radiographicinterpretation information storage unit 132 in the storage device 13,syntax analysis is carried out using morphological analysis or the liketo extract keywords, which are classified as “site”, “symptom”, “diseasename”, “action”, etc.

FIG. 7 shows, by way of example, keyword extraction by the case imagegenerating apparatus 1 according to an embodiment. In the example shownin FIG. 7, through the syntactic analysis of “nodular shadow found inleft lung field, upper lobe; squamous cell carcinoma suspected; workupby HR-CT instructed”, the following keywords have been extracted: “leftlung field, upper lobe” as “site”, “nodular shadow” as “symptom”,“squamous cell carcinoma suspected” as “disease name”, and “workup byHR-CT” as “action”.

Returning to FIG. 2, an information storage unit 203 stores the waveletfeatures calculated in the above-described manner and the extractedkeywords, in the visual word storage unit 133 in the storage device 13.The unit 203 stores the wavelet features and the keywords in associationwith the past images stored in the medical image storage unit 131 in thestorage device 13.

A group generation unit 204 classifies the stored images on the basis ofthe extracted keywords to generate a plurality of groups. While thecategories for classification are not particularly restricted, aplurality of groups are preferably generated by classifying the imagesunder the items used in the syntax analysis, i.e. “site”, “symptom”,“disease name” etc., and/or one or more combinations thereof. One imagemay of course be classified into more than one group.

A centroid vector calculation unit 205 calculates, for each group, acentroid vector of wavelet features of the images corresponding to thekeywords included in that group. More specifically, for each of theimages included in the group, M wavelet features (M is a natural numberof 2 or greater), for example 24 wavelet features, are calculated foreach pixel, and the calculated wavelet features are binarized andconverted to an M-dimensional bit string (M=24).

A frequency distribution vector calculation unit 208 generates ahistogram indicating the frequency distribution of the values of the24-dimensional bit strings obtained through conversion. Such a histogramis generated for all the images included in the group.

FIG. 8 shows an example of a histogram according to one embodiment. Inthis example, 2²⁴ values are taken along the horizontal axis, andfrequency distribution is obtained for the respective values. Then, thefrequency distribution for each image is stored as a frequencydistribution vector, in the frequency distribution information storageunit 134 in the storage device 13.

The centroid vector calculation unit 205 uses the wavelet features andthe calculated frequency distribution vectors of the images included ineach of the groups to calculate a centroid vector of the frequencydistribution vectors for each group, in accordance with the followingexpression (7). More specifically, the centroid vector V_(T) of thefrequency distribution vectors V_(i) (i is the number of images includedin the group) is calculated by dividing the total sum of the frequencydistribution vectors V_(i) by the total sum of the norms (lengths) ofthe frequency distribution vectors V_(i).

$\begin{matrix}{V_{T} = \frac{\sum\limits_{i}V_{i}}{\sum\limits_{i}{V_{i}}}} & (7)\end{matrix}$

Returning to FIG. 2, a spatial distance calculation unit 206 calculatesa spatial distance between the calculated centroid vector and thewavelet feature-based feature vector (frequency distribution vector) ofeach of the images corresponding to the keywords included in the group.The spatial distance calculation unit 206 calculates the spatialdistance as an angle between the centroid vector and the feature vector(frequency distribution vector). More specifically, when the frequencydistribution vector of an image included in the group is represented asV_(i) and the centroid vector is represented as V_(T), then the spatialdistance is calculated as the cosine of the angle φ between the twovectors, i.e. cos φ, in accordance with the following expression (8).

$\begin{matrix}{{\cos \; \varphi} = \frac{\langle{V_{i},V_{T}}\rangle}{{V_{i}} \cdot {V_{T}}}} & (8)\end{matrix}$

In the above expression (8), <V_(i), V_(T)> indicates the inner productof the vectors V_(i) and V_(T), and the denominator indicates theproduct between the norm (length) of the vector V_(i) and the norm ofthe vector V_(T).

Returning to FIG. 2, a representative image storage unit 207 stores theimage the calculated spatial distance of which is the shortest andradiographic interpretation information associated with that image, as arepresentative image of the group, in the case image database 135. Theimage having a shorter spatial distance is closer to the centroid vectorof the images included in the group, and therefore it is suitably usedas a representative image for a case.

It is noted that when a representative image is displayed on the displaydevice 23, feature vectors may be overlaid on the displayed image. FIG.9 shows an example of a representative image display screen used in thecase image generating apparatus 1 according to one embodiment.

As shown in FIG. 9, a past image that has been determined to be closestto the centroid vector is displayed as the representative image and, ofthe wavelet features, those greater than a predetermined value aredisplayed overlaid on the image as the feature vectors. The length ofeach arrow indicates the magnitude of the feature value. The directionof each arrow indicates the one of the eight directions in which thefeature value is greatest. The scales may also be distinguished bycolors, line types, and so on.

FIG. 10 is a flowchart illustrating the processing procedure of the CPU11 in the case image generating apparatus 1 according to one embodiment.Referring to FIG. 10, the CPU 11 in the case image generating apparatus1 calculates wavelet features of a plurality of images that have beentaken and stored in the past (S1001). In the present embodiment, Gaborwavelet features are calculated as the wavelet features.

The CPU 11 extracts keywords included in the radiographic interpretationinformation for the respective stored past images (S1002), and storesthe extracted keywords and the wavelet features calculated in theabove-described manner, in association with the stored past images.

The CPU 11 generates a plurality of groups by classifying the storedimages on the basis of the extracted keywords (S1003). While thecategories for classification are not particularly restricted, aplurality of groups are preferably generated by classifying the imagesunder the items used in the syntax analysis, i.e. “site”, “symptom”,“disease name”, etc., and/or one or more combinations thereof. One imagemay of course be classified into more than one group.

The CPU 11 calculates, for each group, a centroid vector as a featurevector, on the basis of the wavelet features of the images included inthe group (S1004). The CPU 11 calculates a spatial distance between thecalculated centroid vector and the wavelet feature-based feature vector(frequency distribution vector) of each of the images corresponding tothe keywords included in the group (S1005).

The CPU 11 sets a predetermined value as a minimum value (S1006), andselects one of the images included in the group (S1007). The CPU 11determines whether the spatial distance calculated for the selectedimage is smaller than the minimum value (S1008).

If the CPU 11 determines that it is smaller than the minimum value (YESin S1008), the CPU 11 stores the spatial distance as the minimum value(S1009). If the CPU 11 determines that the spatial distance calculatedis not smaller than the minimum value (NO in S1008), the CPU 11 skipsS1009. The CPU 11 determines whether all the images have been selected(S1010). If the CPU 11 determines that there is an image yet to beselected (NO in S1010), the CPU 11 selects a next image (S1011). Theprocess then returns to S1008, and the above-described processing isrepeated.

If the CPU 11 determines that all the images have been selected (YES inS1010), the CPU 11 stores the image corresponding to the spatialdistance stored as the minimum value and the radiographic interpretationinformation corresponding thereto, as a representative image, in thecase image database 135 (S1012).

As described above, according to the present embodiment, waveletfeatures indicating the features of stored medical images are used tocalculate feature vectors for the respective images, and from thesefeature vectors, a centroid vector is calculated as a feature vector foreach case. Among the stored medical images, an image having the featurevector with the shortest spatial distance from the centroid vector isstored as a representative image. Therefore, the image indicating atypical case can be used as a guideline of the case, allowing adiagnosis to be made for a patient at a certain level of quality,without being affected by experience or expertise of each doctor.

It is noted that the present invention is not restricted to theabove-described embodiment; a variety of modifications and improvementsare possible within the scope of the present invention.

The present invention may be a system, a method, and/or a computerprogram product. The computer program product may include a computerreadable storage medium (or media) having computer readable programinstructions thereon for causing a processor to carry out aspects of thepresent invention.

The computer readable storage medium can be a tangible device that canretain and store instructions for use by an instruction executiondevice. The computer readable storage medium may be, for example, but isnot limited to, an electronic storage device, a magnetic storage device,an optical storage device, an electromagnetic storage device, asemiconductor storage device, or any suitable combination of theforegoing. A non-exhaustive list of more specific examples of thecomputer readable storage medium includes the following: a portablecomputer diskette, a hard disk, a random access memory (RAM), aread-only memory (ROM), an erasable programmable read-only memory (EPROMor Flash memory), a static random access memory (SRAM), a portablecompact disc read-only memory (CD-ROM), a digital versatile disk (DVD),a memory stick, a floppy disk, a mechanically encoded device such aspunch-cards or raised structures in a groove having instructionsrecorded thereon, and any suitable combination of the foregoing. Acomputer readable storage medium, as used herein, is not to be construedas being transitory signals per se, such as radio waves or other freelypropagating electromagnetic waves, electromagnetic waves propagatingthrough a waveguide or other transmission media (e.g., light pulsespassing through a fiber-optic cable), or electrical signals transmittedthrough a wire.

Computer readable program instructions described herein can bedownloaded to respective computing/processing devices from a computerreadable storage medium or to an external computer or external storagedevice via a network, for example, the Internet, a local area network, awide area network and/or a wireless network. The network may comprisecopper transmission cables, optical transmission fibers, wirelesstransmission, routers, firewalls, switches, gateway computers and/oredge servers. A network adapter card or network interface in eachcomputing/processing device receives computer readable programinstructions from the network and forwards the computer readable programinstructions for storage in a computer readable storage medium withinthe respective computing/processing device.

Computer readable program instructions for carrying out operations ofthe present invention may be assembler instructions,instruction-set-architecture (ISA) instructions, machine instructions,machine dependent instructions, microcode, firmware instructions,state-setting data, or either source code or object code written in anycombination of one or more programming languages, including an objectoriented programming language such as Smalltalk, C++ or the like, andconventional procedural programming languages, such as the “C”programming language or similar programming languages. The computerreadable program instructions may execute entirely on the user'scomputer, partly on the user's computer, as a stand-alone softwarepackage, partly on the user's computer and partly on a remote computeror entirely on the remote computer or server. In the latter scenario,the remote computer may be connected to the user's computer through anytype of network, including a local area network (LAN) or a wide areanetwork (WAN), or the connection may be made to an external computer(for example, through the Internet using an Internet Service Provider).In some embodiments, electronic circuitry including, for example,programmable logic circuitry, field-programmable gate arrays (FPGA), orprogrammable logic arrays (PLA) may execute the computer readableprogram instructions by utilizing state information of the computerreadable program instructions to personalize the electronic circuitry,in order to perform aspects of the present invention.

Aspects of the present invention are described herein with reference toflowchart illustrations and/or block diagrams of methods, apparatus(systems), and computer program products according to embodiments of theinvention. It will be understood that each block of the flowchartillustrations and/or block diagrams, and combinations of blocks in theflowchart illustrations and/or block diagrams, can be implemented bycomputer readable program instructions.

These computer readable program instructions may be provided to aprocessor of a general purpose computer, special purpose computer, orother programmable data processing apparatus to produce a machine, suchthat the instructions, which execute via the processor of the computeror other programmable data processing apparatus, create means forimplementing the functions/acts specified in the flowchart and/or blockdiagram block or blocks. These computer readable program instructionsmay also be stored in a computer readable storage medium that can directa computer, a programmable data processing apparatus, and/or otherdevices to function in a particular manner, such that the computerreadable storage medium having instructions stored therein comprises anarticle of manufacture including instructions which implement aspects ofthe function/act specified in the flowchart and/or block diagram blockor blocks.

The computer readable program instructions may also be loaded onto acomputer, other programmable data processing apparatus, or other deviceto cause a series of operational steps to be performed on the computer,other programmable apparatus or other device to produce a computerimplemented process, such that the instructions which execute on thecomputer, other programmable apparatus, or other device implement thefunctions/acts specified in the flowchart and/or block diagram block orblocks.

The flowchart and block diagrams in the Figures illustrate thearchitecture, functionality, and operation of possible implementationsof systems, methods, and computer program products according to variousembodiments of the present invention. In this regard, each block in theflowchart or block diagrams may represent a module, segment, or portionof instructions, which comprises one or more executable instructions forimplementing the specified logical function(s). In some alternativeimplementations, the functions noted in the block may occur out of theorder noted in the figures. For example, two blocks shown in successionmay, in fact, be executed substantially concurrently, or the blocks maysometimes be executed in the reverse order, depending upon thefunctionality involved. It will also be noted that each block of theblock diagrams and/or flowchart illustration, and combinations of blocksin the block diagrams and/or flowchart illustration, can be implementedby special purpose hardware-based systems that perform the specifiedfunctions or acts or carry out combinations of special purpose hardwareand computer instructions.

While the invention has been described with reference to exemplaryembodiments, it will be understood by those skilled in the art thatvarious changes may be made and equivalents may be substituted forelements thereof without departing from the scope of the invention. Inaddition, many modifications may be made to adapt a particular system,device or component thereof to the teachings of the invention withoutdeparting from the essential scope thereof. Therefore, it is intendedthat the invention not be limited to the particular embodimentsdisclosed for carrying out this invention, but that the invention willinclude all embodiments falling within the scope of the appended claims.Moreover, the use of the terms first, second, etc. do not denote anyorder or importance, but rather the terms first, second, etc. are usedto distinguish one element from another.

The terminology used herein is for the purpose of describing particularembodiments only and is not intended to be limiting of the invention. Asused herein, the singular forms “a”, “an” and “the” are intended toinclude the plural forms as well, unless the context clearly indicatesotherwise. It will be further understood that the terms “comprises”and/or “comprising,” when used in this specification, specify thepresence of stated features, integers, steps, operations, elements,and/or components, but do not preclude the presence or addition of oneor more other features, integers, steps, operations, elements,components, and/or groups thereof.

The corresponding structures, materials, acts, and equivalents of allmeans or step plus function elements in the claims below, if any, areintended to include any structure, material, or act for performing thefunction in combination with other claimed elements as specificallyclaimed. The description of the present invention has been presented forpurposes of illustration and description, but is not intended to beexhaustive or limited to the invention in the form disclosed. Manymodifications and variations will be apparent to those of ordinary skillin the art without departing from the scope and spirit of the invention.The embodiments were chosen and described in order to best explain theprinciples of the invention and the practical application, and to enableothers of ordinary skill in the art to understand the invention forvarious embodiments with various modifications as are suited to theparticular use contemplated.

What is claimed is:
 1. A method of generating a representative imagerepresenting a case and radiographic interpretation information for eachcase from medical images based on past cases, comprising: calculating,by a computer system, wavelet features of a plurality of images thathave been taken and stored in the past; extracting, by the computersystem, a keyword included in radiographic interpretation informationfor each of the stored images; storing, by the computer system, thecalculated wavelet features and the extracted keywords in associationwith the respective stored images; classifying, by the computer system,the stored images on the basis of the extracted keywords to generate aplurality of groups; calculating, by the computer system for each of thegenerated groups, a centroid vector of wavelet feature-based featurevectors of the respective images corresponding to the keywords includedin that group; calculating, by the computer system for each of thegroups, a spatial distance between the calculated centroid vector andeach of the wavelet feature-based feature vectors of the respectiveimages corresponding to the keywords included in that group; andstoring, by the computer system for each of the groups, an image forwhich the calculated spatial distance is the shortest and theradiographic interpretation information associated with the image as arepresentative image of that group.
 2. The method of claim 1, whereinthe wavelet features are two-dimensional Gabor wavelet features.
 3. Themethod of claim 1, further comprising: calculating frequencydistribution vectors for all images by calculating M said waveletfeatures for each image and binarizing the respective wavelet featuresfor conversion into an M-dimensional bit string, wherein the spatialdistance is calculated as an angle between the centroid vector and eachof the calculated frequency distribution vectors, and wherein M is anatural number of 2 or greater.
 4. An apparatus for generating arepresentative image representing a case and radiographic interpretationinformation for each case from medical images based on past cases,comprising: a wavelet feature calculation unit configured to calculatewavelet features of a plurality of images that have been taken andstored in the past; a keyword extraction unit configured to extract akeyword included in radiographic interpretation information for each ofthe stored images; an information storage unit configured to store thecalculated wavelet features and the extracted keywords in associationwith the respective stored images; a group generation unit configured toclassify the stored images on the basis of the extracted keywords togenerate a plurality of groups; a centroid vector calculation unitconfigured to calculate, for each of the generated groups, a centroidvector of wavelet feature-based feature vectors of respective imagescorresponding to the keywords included in that group; a spatial distancecalculation unit configured to calculate, for each of the groups, aspatial distance between the calculated centroid vector and each of thewavelet feature-based feature vectors of the respective imagescorresponding to the keywords included in that group; and arepresentative image storage unit configured to store, for each of thegroups, the image for which the calculated spatial distance is theshortest and the radiographic interpretation information associated withthe image as a representative image of that group.
 5. The apparatus ofclaim 4, wherein the wavelet feature calculation unit calculatestwo-dimensional Gabor wavelet features.
 6. The apparatus of claim 4,further comprising: a frequency distribution vector calculation unit forcalculating frequency distribution vectors for all images by calculatingM said wavelet features for each image and binarizing the respectivewavelet features for conversion into an M-dimensional bit string,wherein the spatial distance calculation unit calculates the spatialdistance as an angle between the centroid vector and each of thecalculated frequency distribution vectors, and wherein M is a naturalnumber of 2 or greater.
 7. A computer program product, comprising: acomputer-readable storage device; and program code embodied on thecomputer-readable storage device, wherein the program code, whenexecuted by a processor, configures the processor to: calculate waveletfeatures of a plurality of images that have been taken and stored in thepast; extract a keyword included in radiographic interpretationinformation for each of the stored images; store the calculated waveletfeatures and the extracted keywords in association with the respectivestored images; classify the stored images on the basis of the extractedkeywords to generate a plurality of groups; calculate, for each of thegenerated groups, a centroid vector of wavelet feature-based featurevectors of respective images corresponding to the keywords included inthat group; calculate, for each of the groups, a spatial distancebetween the calculated centroid vector and each of the waveletfeature-based feature vectors of the respective images corresponding tothe keywords included in that group; and store, for each of the groups,the image in which the calculated spatial distance is the shortest andthe radiographic interpretation information associated with the image asa representative image of that group.
 8. The computer program of claim7, wherein the wavelet features are two-dimensional Gabor waveletfeatures.
 9. The computer program of claim 7, wherein the program code,when executed by the processor, further configures the processor to:calculate frequency distribution vectors for all images by calculating Msaid wavelet features for each image and binarizing the respectivewavelet features for conversion into an M-dimensional bit string,wherein M is a natural number of 2 or greater; and calculate the spatialdistance as an angle between the centroid vector and each of thecalculated frequency distribution vectors.